Deep Learning Exam

Perceptron

The symbol represents the prediction.
is the function that defines the line of the perceptron classifier.

Example

The formula for a 2 word classifier is , where:

Activation Function "step"

In this instance, a positive result would be turned to 1 by the activation function, and a negative one to 0.

We are talking about a method to train the perceptron.
There are 2 cases:

If a point is correctly classified: do nothing.
If a point is incorrectly classified, move the line towards that point by changing the weights and bias.

Example

We represent each word in the dictionary with a vector(of variable dimensions).

Notation

The expression computes the vector difference between and , resulting in a new vector that points from to .
denote the "norm" of a vector, AKA the vector's length.

Another way to measure how similar two vectors are, but this time it's based on the angle and not the vectors length.

Example

We build a neural network to solve the following problem:

The following is the architecture for skipgram, which has all the possible words in the first and last layer, each word has its own neuron:

We just make a sliding window on a text and we label as 1 the words that appear and 0 the words that don't appear.

Lil problem